Retrieving Images with Generated Textual Descriptions

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper presents a novel remote sensing (RS) image retrieval system that is defined based on generation and exploitation of textual descriptions that model the content of RS images. The proposed RS image retrieval system is composed of three main steps. The first one generates textual descriptions of the content of the RS images combining a convolutional neural network (CNN) and a recurrent neural network (RNN) to extract the features of the images and to generate the descriptions of their content, respectively. The second step encodes the semantic content of the generated descriptions using word embedding techniques able to produce semantically rich word vectors. The third step retrieves the most similar images with respect to the query image by measuring the similarity between the encoded generated textual descriptions of the query image and those of the archive. Experimental results on RS image archive composed of RS images acquired by unmanned aerial vehicles (UAVs) are reported and discussed

Characterizing the spatiotemporal distribution of meteorological drought as a response to climate variability: The case of rift valley lakes basin of Ethiopia

Climate variability and recurrent meteorological droughts frequently affect the rain-dependent Ethiopian agriculture, where the rift valley lakes basin is one of the most drought-prone regions in the country. The aim of this study was to evaluate climate variability and characterize the spatiotemporal distribution of meteorological droughts using a merged satellite-gauge rainfall across the major agroecological zones (AEZs) of the rift valley lakes basin. To this end, coefficient of variation (CV) and standardized rainfall anomaly (SRA) were used to evaluate rainfall variability; Mann-Kendell test was used to examine trends of temperature and rainfall; and a grid-rainfall based standardized precipitation index (SPI) was used to assess the spatiotemporal distribution and severity of meteorological droughts. The SPI was computed for 37 years over 1981–2017 at 3-month and 4-month timescales for the bimodal rainy seasons. Finally, a higher inter-annual and spatial variability of rainfall and frequent meteorological droughts were found across the basin. Compared to the nationally documented historical drought years in the country, more frequent drought events were found in this basin, signifying its higher vulnerability to climate variability. As a result, between 1981 and 2017, the basin has partially experienced at least a moderate drought intensity on average every 1.68 and 1.76 years during the 'Belg' and 'Kiremt' season, respectively. Drought frequency was higher at the 'Kolla' AEZ, characterized by the highest CV of rainfall. Furthermore, these frequent droughts were accompanied by significant rising trends in monthly temperature. Such a warming trend, in this inherently warm area, coupled with expected global climate change scenarios could further aggravate drought conditions in the future. Moreover, the spatiotemporal distribution of drought events was found to be variable between and within AEZs in the basin so that more localized drought adaptation strategies could help to alleviate potential impacts. Thus, the drought history of each agroecological zone and the spatiotemporal distributions of recent droughts, this study has delivered, could enhance the awareness of concerned decision makers in tracing frequently affected locations, which could in turn enable them to design and implement improved water management techniques as a means of drought mitigation strategy. Keywords: Climate variability, Drought, Mann-Kendall test, Merged satellite-gauge rainfall, Rift valley lakes basin, SP

MaxEnt-based modeling of suitable habitat for rehabilitation of Podocarpus forest at landscape-scale

AbstractModeling the current distribution and predicting suitable habitats of threatened species support proper planning processes for conservation and restoration. The aim of this study was thus to model the actual distribution and predict environmentally suitable habitats for Podocarpus falcatus, a locally threatened native tree species in Ethiopia. To realize this objective, species' presence samples, BIOCLIMATIC, and topographic predictors were combined to run a MaxEnt model. Finally, a model-generated habitat suitability map was produced with AUC accuracy of 0.783. Among the variables used for modeling, elevation range was found to be a key predictor of Podocarpus distribution, followed by precipitation of the driest quarter and isothermality. An extensive area (> 48%) of the studied landscape has been predicted to be environmentally suitable for the target species. However, only a small portion open-land area is practically available for rehabilitation since the area has been intensively cultivated to support the densely inhabited population. Therefore, potential areas for a small-scale plantation of Podocarpus trees remain to be pocket sites in religious places and around farmers' homesteads. So far, many farmers in this area have demonstrated a successful experience of growing this degraded native tree species. Thus, encouraging privately owned small-scale plantations could enhance rehabilitation and more sustainable conservation of the locally threatened native tree species

Toward Remote Sensing Image Retrieval Under a Deep Image Captioning Perspective

The performance of remote sensing image retrieval (RSIR) systems depends on the capability of the extracted features in characterizing the semantic content of images. Existing RSIR systems describe images by visual descriptors that model the primitives (such as different land-cover classes) present in the images. However, the visual descriptors may not be sufficient to describe the high-level complex content of RS images (e.g., attributes and relationships among different land-cover classes). To address this issue, in this article, we present an RSIR system that aims at generating and exploiting textual descriptions to accurately describe the relationships between the objects and their attributes present in RS images with captions (i.e., sentences). To this end, the proposed retrieval system consists of three main steps. The first step aims to encode the image visual features and then translate the encoded features into a textual description that summarizes the content of the image with captions. This is achieved based on the combination of a convolutional neural network with a recurrent neural network. The second step aims to convert the generated textual descriptions into semantically meaningful feature vectors. This is achieved by using the recent word embedding techniques. Finally, the last step estimates the similarity between the vectors of the textual descriptions of the query image and those of the archive images, and then retrieve the most similar images to the query image. Experimental results obtained on two different datasets show that the description of the image content with captions in the framework of RSIR leads to an accurate retrieval performance.EC/H2020/759764/EU/Accurate and Scalable Processing of Big Data in Earth Observation/BigEart

Capsule Networks for Object Detection in UAV Imagery

Recent advances in Convolutional Neural Networks (CNNs) have attracted great attention in remote sensing due to their high capability to model high-level semantic content of Remote Sensing (RS) images. However, CNNs do not explicitly retain the relative position of objects in an image and, thus, the effectiveness of the obtained features is limited in the framework of the complex object detection problems. To address this problem, in this paper we introduce Capsule Networks (CapsNets) for object detection in Unmanned Aerial Vehicle-acquired images. Unlike CNNs, CapsNets extract and exploit the information content about objects’ relative position across several layers, which enables parsing crowded scenes with overlapping objects. Experimental results obtained on two datasets for car and solar panel detection problems show that CapsNets provide similar object detection accuracies when compared to state-of-the-art deep models with significantly reduced computational time. This is due to the fact that CapsNets emphasize dynamic routine instead of the depth.EC/H2020/759764/EU/Accurate and Scalable Processing of Big Data in Earth Observation/BigEart

Hierarchical ownership and deterministic watermarking of digital images via polynomial interpolation

This paper presents a novel method for the secure management of digital images formulated within the mathematical theory of polynomial interpolation. As main innovative features, our approach is based on a hierarchical joint ownership of the image by a trusted layered authority and on a deterministic watermarking procedure, embedding a short meaningful or random signature into the image. Experimental results show that the inserted signature can almost always be fully recovered even in presence of a reasonable amount of image degradation due to image processing operators, such as filtering, geometric distortions and compression

Three Dimensional Electromagnetic Sub-Surface Sensing by Means of a Multi-Step SVM-Based Classification Technique

In this paper, the classification approach is extended from 2D to the three‐dimensional(3D) case carefully addressing the increased complexity issue by means of an effective multi‐step strategy. As a matter of fact, by iteratively processing the training dataset (without requiring an extra amount of measurements), the proposed method is aimed at improving the spatial resolution of the original classification technique [6] even though dealing with a more complex problem. The effectiveness of the proposed approach has been preliminary assessed through a set of numerical experiments also in correspondence with blurred data and some representative results are shown in the following. This is the author's version of the final version available at IEEE

Active learning methods for electrocardiographic signal classification

In this paper, we present three active learning strategies for the classification of electrocardiographic (ECG) signals. Starting from a small and suboptimal training set, these learning strategies select additional beat samples from a large set of unlabeled data. These samples are labeled manually, and then added to the training set. The entire procedure is iterated until the construction of a final training set representative of the considered classification problem. The proposed methods are based on support vector machine classification and on the: 1) margin sampling; 2) posterior probability; and 3) query by committee principles, respectively. To illustrate their performance, we conducted an experimental study based on both simulated data and real ECG signals from the MIT-BIH arrhythmia database. In general, the obtained results show that the proposed strategies exhibit a promising capability to select samples that are significant for the classification process, i.e., to boost the accuracy of the classification process while minimizing the number of involved labeled samples

Genetic algorithm-based method for mitigating label noise issue in ECG signal classification

Classification of electrocardiographic (ECG) signals can be deteriorated by the presence in the training set of mislabeled samples. To alleviate this issue we propose a new approach that aims at assisting the human user (cardiologist) in his/her work of labeling by removing in an automatic way the training samples with potential mislabeling problems. The proposed method is based on a genetic optimization process, in which each chromosome represents a candidate solution for validating/invalidating the training samples. Moreover, the optimization process consists of optimizing jointly two different criteria, which are the maximization of the statistical separability among classes and the minimization of the number of invalidated samples. Experimental results obtained on real ECG signals extracted from the MIT-BIH arrhythmia database confirm the effectiveness of the proposed solution